Evolutionarily stable strategy

An Evolutionarily stable strategy is a behavioral policy that, once adopted by most members of a population, cannot be bettered by an alternative. Developed in the book through game-theoretic models of conflict, it explains stability without assuming that individuals consciously calculate, or that a population is organized for the common good. Its decisive point is that a stable outcome need not be the best outcome for everyone: it persists because an individual cannot gain by breaking from it.

Definition and evolutionary logic

A strategy is a pre-programmed behavioral policy: a convenient way to describe an animal as acting according to instructions, not a claim that it consciously reasons them out. The animal is treated as a survival machine whose behavior is controlled by a pre-programmed system. The description may take the form of a conditional rule, such as attacking, pursuing a fleeing opponent, and retreating if the opponent retaliates. The words make a pattern legible; they do not imply deliberate choice.

A strategy is evolutionarily stable when, if most of the population adopts it, no alternative strategy can invade. The result depends on what others do: a policy that succeeds against one prevailing population may fare badly when the population changes. Selection therefore favors not an abstractly best move but one that holds up against the strategies it is likely to encounter. Environmental change may produce a temporary period of instability, even oscillation, but an established stable strategy resists deviations that would otherwise spread.

This argument is framed as a complex, potentially unconscious cost-benefit calculation. Fighting can bring a valuable resource, but it also carries risks, consumes time and energy, and may leave a victor unable to benefit from winning. Waiting to grow stronger can improve the chances of a later contest. The calculations need not occur in the animal’s mind: the game-theoretic account describes how policies with different consequences fare under selection. John Maynard Smith, working with G. R. Price and G. A. Parker, gave this analysis a clear formulation; the concept is also traced to W. D. Hamilton and R. H. MacArthur.

The hawk-and-dove model

The simplest model compares two strategies, Hawk and Dove. A Hawk fights without restraint and retreats only when seriously injured; a Dove threatens conventionally but does not inflict injury. In the model, neither opponent can be identified in advance, and individuals have no memory of earlier encounters that could guide their behavior. A Hawk meeting a Dove wins the contest without injury, whereas a Hawk meeting another Hawk risks a costly fight. Two Doves posture until one gives way, without either being injured.

The changing payoffs show why the success of a strategy depends on its frequency. When a mutant Hawk is rare, it meets Doves and receives an average payoff of +50, compared with +15 for the Doves. Hawk genes consequently spread. But if Hawks become common, Hawk-on-Hawk contests make the average Hawk payoff −25: each Hawk is expected to win half such encounters at +50 and lose half at −100. A Dove in that Hawk-dominated population loses contests but avoids injury, with an average payoff of zero; Dove genes can then increase.

The model does not require a continuing swing from one majority to the other. With the stated, arbitrary points system, the stable composition is seven-twelfths Hawks and five-twelfths Doves, or a 7:5 ratio. At that point the two strategies have equal average payoffs, so selection favors neither. If the Hawk share rises, Doves gain an advantage and the balance moves back. Small fluctuations around the stable point are possible without overturning the equilibrium.

Stability is not collective welfare

The equilibrium can resemble group selection because it describes a population ratio that tends to recover when disturbed. But the explanation is different: stability follows from the payoffs to individuals facing one another, not from competition in which successful groups replace unsuccessful ones. At the 7:5 equilibrium, the average payoff is 6¼ for either strategy. In an all-Dove population, by contrast, every individual would receive 15. A group-selection account might therefore predict that populations of Doves should prevail because each member does better than in the mixed equilibrium.

The all-Dove arrangement, however, can be invaded. A rare Hawk among Doves gains so much by exploiting their restraint that Hawk genes would spread and break the arrangement. The equilibrium is stable not because it gives participants the greatest possible benefit, but because it is resistant to internal treachery. The contrast separates a mutually advantageous arrangement from one that can endure when an individual has an incentive to defect.

Humans can sometimes sustain pacts that are advantageous to every participant because conscious foresight lets them recognize a long-term interest in obeying the rules. Even these arrangements remain vulnerable to short-term temptation. The book’s example is price-fixing among garage owners: owners may benefit from maintaining an artificially high common petrol price, yet an individual can make a quick gain by cutting prices. Neighbours then follow, the agreement collapses, and a new pact may later form. If such consciously maintained agreements are precarious, the account argues, group-benefit strategies are harder still to explain in wild animals whose behavior is shaped by selection. Stable strategies are therefore expected to be widespread.

Conditional strategies and aggression

The model can be extended beyond the two simple alternatives. A retaliator begins as a Dove, using a conventional threat, but fights back if attacked. Against a Dove or another retaliator, it maintains the non-escalated pattern; against a Hawk, it responds as a Hawk. Its policy is conditional on the opponent’s conduct. A Bully acts like a Hawk until somebody hits back, then retreats. A Prober-retaliator resembles a retaliator but occasionally tests an opponent with a brief escalation: it persists if the opponent does not fight back, returns to conventional threat if challenged, and retaliates if attacked.

When five strategies are placed together in a computer simulation, retaliator is the only one that emerges as evolutionarily stable; Prober-retaliator comes close. Dove is vulnerable to invasion by Hawk and Bully, Hawk to Dove and Bully, and Bully to Hawk. In a population of retaliators, none of the alternatives does better, although Dove does equally well. If Doves increase substantially, Prober-retaliators, as well as Hawks and Bullies, gain an advantage over retaliators. In a population of Prober-retaliators, only retaliator does better, and then only slightly. The predicted outcome is thus a predominance of retaliators and Prober-retaliators, perhaps with a small Dove minority and gentle shifts in their proportions. The population need not consist of individuals who each play a single fixed strategy; an individual may express a mixture of strategies.

This conditional structure helps explain aggression that is neither unrestrained nor wholly peaceful—the book’s “gloved fist” pattern. The balance depends on the costs and rewards of a particular contest. For an Elephant seal, victory may bring near-monopoly access to a large harem, making the prize exceptionally valuable and serious fighting more likely. For a Great tit feeding nestlings, time spent in a contest may be especially costly: it needs to catch an average of one prey item every thirty seconds. The strategy’s consequences therefore depend on the ecological stakes, rather than on a universal ranking of aggressive and restrained behavior.

Territorial behavior and a broader application

The same reasoning applies where an interaction has an asymmetry between resident and intruder. Residents may know the local terrain better, while an intruder may be more tired after moving into the area. A strategy in which the resident wins and the intruder retreats can therefore be a stable outcome. The reverse arrangement—intruders winning and residents retreating—is described as paradoxical: individuals would be selected to avoid being residents, encouraging constant movement and tending to eliminate the resident category itself. Where selection instead rewards remaining on a particular patch, individuals tend to stay and appear to defend it. Such behavior is recognizable as territorial defense.

An experiment by Niko Tinbergen demonstrates how a change in the asymmetry changes the behavior. Two male Sticklebacks in a tank each built a nest and defended the surrounding area. Tinbergen placed each fish in a large glass test tube and held the tubes beside one another so that the males could not fight. Near one male’s nest, that male took the attacking posture and the other tried to retreat. When Tinbergen moved the tubes to the other nest’s territory, their roles reversed. The same two fish changed which one attacked and which withdrew as the location changed. The behavior fit a conditional policy: attack when resident, retreat when intruding. Territorial defense, on this account, need not require a separate group-level benefit; it can arise as a stable response to the usual asymmetry in arrival at a patch of ground.

The concept also extends from visible interactions between whole animals to selection among genes. The gene pool is described as the long-term environment of a gene, in which a gene’s success is assessed against the other genes currently present. A gene pool is evolutionarily stable when no new gene can invade it. Most new genes arising through mutation, reassortment, or immigration are penalized by selection and disappear; occasionally one succeeds, spreads, and produces a transitional period before a new stable set is established. There may be multiple stable points, with evolution proceeding through shifts from one plateau to another rather than as a smooth ascent.

That population-level appearance does not mean the population acts as a single self-regulating unit. Selection operates at the level of individual genes, whose success depends on performance against the prevailing gene pool. Interactions between visible animal strategies thus provide an accessible model for a much larger set of interactions among genes, most of which occur within bodies, especially in developing cells and embryos. Well-integrated bodies emerge from the compatibility of genes in an evolutionarily stable set. In this way, the strategy concept connects behavioral conflict to the wider gene-centered account: persistence reflects resistance to invasion, not conscious purpose or collective design.